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ABSTRACT 


This preliminary study developed the concept map tool “BR-Map” using learning logs on eBook viewer, and investigated 
the relationships between self-regulated learning (SRL) awareness, learning behaviors (usage of BR-Map, and 
one-minute paper and report submission), and learning performance. Psychometric data and learning logs were collected 
in the lecture course, and their relationships were analyzed using Spearman’s correlation analysis. The results indicated 
that awareness of intrinsic value, use of cognitive learning strategies, and self-regulation had significant correlations with 
the usage of BR-Map. The awareness of cognitive learning strategies had significant correlation with standard deviation 
of one-minute paper submission hours. With regard to relationships between the BR-Map usage and learning behaviors, 
the relationships between the usage of BR-Map and one-minute paper submissions, which was a regularly weekly 
assigned task, were found. 
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1. INTRODUCTION 


1.1 Cognitive Learning 


Cognitive learning research has been conducted for a long time in the field of educational technology 
research. Not only input but also consideration of, and interaction with, learning materials deepen 
information processing in both the mind and brain, and promote learning outcomes. Cognitive learning tools 
play an important role in enhancing learning outcomes, according to many previous researches (e.g., Leopold 
and Leutner, 2012). Concept maps are an effective cognitive learning tool for the enhancement of learning 
outcomes. Previous research (e.g., Clariana, et al, 2013; Yamada, 2010) indicated that a concept map makes 
learners aware of learning objects and the presence of peers. Concept maps promote cognitive learning 
performance and strategies (Fiorella and Mayer, 2013, 2017). 

Perry and Winne (2006) evaluated the effects of the integrated cognitive learning tool, “gStudy,” which 
was developed based on a self-regulated learning (SRL) model that centered on meta-cognitive skills. 
Learners become aware of academic strengths and weaknesses through metacognition. gStudy records 
learner’s learning behaviors, promotes the cognitive learning process, and gives learners feedback. The 
concept map seems to be effective not only in improving learning performance but also the enhancement of 
metacognition. As further research, tracing and visualizing the learning process including the SRL process 
from input to consideration of concept maps and other learning support systems is desirable for creating an 
effective learning environment. 


1.2 SRL and Learning Analytics 


Self-regulated learning (SRL) is one of the important viewpoints for understanding learning behaviors. SRL 
is the active learning process used to regulate and monitor learning cognition, motivation, and behavior, and 
to set personal learning goals, including social aspects (Wolters, Pintrich, & Karabenick, 2003; Schunk and 
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Zimmerman, 2008). SRL also relates to metacognition (Schunk, 2008) and information processing (Winne 
and Hadwin, 1998). SRL seems to be a useful concept for understanding learners’ learning features. The 
effects of SRL seem to be different for high- and low-performers. Schunk and Zimmerman (1998) further 
compared the learning behaviors of novice and expert SRL learners. Their results indicated that skillful 
learners controlled their learning process—such as making their learning plan, monitoring and reflection with 
their metacognition—and then they felt high self-efficacy, and had high internal motivation and learning 
performance. 

Advances in Information and Communication Technology (ICT) can be of benefit to both learners and 
teachers to enhance SRL awareness and skills. When using ICT, learners can control when, what, and how 
they learn, without restrictions of time, learning space, and printed materials (Cunningham and Billingsley, 
2003). Greene and Azevedo (2009) suggested 13 indicators of SRL in the context of computer-based learning 
such as help-seeking, expectation of adequacy of information, time and effort spent in planning. Recent 
research trends are focusing on the relationships between learning performance and SRL. Winne and his 
research colleague (2006) developed “gStudy” with a log analyzer, which constituted an early research about 
SRL in terms of learning analytics. Learning analytics is defined as “to clarify education and learning 
environment improvement using various data such as logs about learners and learning environment, with 
information processing methods” (e.g., Ifenthalar, 2015; Ogata et al, 2015). Goda et al (2013) suggested that 
SRL factors are useful to predict learning performance, and their successive study (2015) suggested that 
high-level SR learners can control and manage their learning plan in the context of their everyday lives, using 
a blended learning environment with ICT. Azevedo et al. (2017) suggested a framework for visualizing SRL 
awareness using multimodal data in e-learning settings. Yamada et al. (2017) suggested that the use of 
cognitive learning strategies—such as annotation as well as appropriate reading time for learning 
materials—play an important role in enhancing SRL awareness. Using ICT, learning behaviors that 
contribute to enhancing SRL awareness can be analyzed to support learning from the perspective of cognitive 
learning in the flow from input to consideration. This study aims to develop a concept map, “BR-Map,” using 
learning logs stored on an ebook viewer that plays an important role in input, and investigate as a preliminary 
research the relationship between the usage of the concept map and SRL. 


2. METHODS 


2.1 Subjects and Course 


Forty-four university students participated in this research. The course consisted of eight classes (one per 
week). The main learning object was to understand educational theories, principles, and history. There were 
two criteria for the grade: submitting a one-minute paper after every class, and a report. Students had to 
submit the one-minute paper within a day for a normal grade, but the teacher would accept it one day late 
(in such cases, the score would be reduced by half). The one-minute paper had to contain an abstract of the 
class and a discussion. The teacher explained the report themes three weeks before the submission deadlines. 
Students were required to submit the one-minute papers and reports on LMS. 


2.2 BR-Map 


BR-Map is a concept map tool using logs stored on an ebook viewer, “BookRoll,” (Ogata et al, 2017) 
displayed in Figure 1. This is a simple and normal concept map tool with an interface as displayed in Figure 
2, but BR-Map uses the logs on the ebook viewer. The usage flow of BR-Map is as follows: 1) The learner 
reads an ebook on BookRoll, 2) Learners highlight part(s) or attach memos on the ebook, 3) Learners open 
BR-Map, 4) BR-Map reads the logs of highlight(s) and memo(s) from the BookRoll database, 5) BR-Map 
lists all logs of highlight(s) and memos and displays them as objects on the left pane, 6) Learners click the 
object on the left pane, and drag-and-drop it on the right pane—the “concept map area,” 7) Learners make a 
concept map by connecting objects using an “arrow”. BR-Map allows learners to make many concept maps, 
using the “tab” function, and to save concept maps as a .png file. BR-Map was developed as a Moodle 
plug-in. The teacher adds one BR-Map plugin on a section in their course. BR-Map reads all the highlights 
and memo logs of all ebooks in the course. 
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BR-Map consists of two parts, the frontend and the server end. The frontend was developed using 
HTML/CSS and JavaScript using libraries including jsPlumb (for arrow presentation), html2canvas (for 
concept map presentation), canvg-browser (convert concept map to picture format (png)), EventBus (for 
event management), download.js (for download function), and jQuery. The server side consists of two 
servers—a web server using nginx 1.12 and a database server MySQL 5.7. Moodle 2.8.5 and PHP 5.6 were 
installed on the web server. 


2.3 Data Collection 


Students were asked to answer the motivational strategies for learning questionnaire (MSLQ) (Pintrich and 
DeGroot, 1990). The MSLQ, which consists of five factors (self-efficacy (SE), internal value (IV), cognitive 
strategies (CS), self-regulation (SR), and test anxiety (TA); 44 items in all, rated on a seven-point Likert 
scale), was used for the subjective evaluation of learners’ SRL skills (see appendix). Students were asked to 
complete the MSLQ in the third class and again in the last class. The second method of data collection was 
the concept maps. The number of objects and links on the concept map of each learner were counted. The 
third method was the log of submission times of the one-minute papers and report. The submission time 
increased the earlier a student submitted the assignment. For example, if a student submitted the one-minute 
paper one hour before the deadline, the submission time was 1; if a student submitted the regular report 100 
hours before deadline, submission time was 100. The final method was to measure report quality. A teacher 
evaluated the report quality as a score in the range of 0 to 40. 
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Figure 1. Interface of “BookRoll” (Ogata et al, 2017) 
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Figure 2. Interface of “BR-Map” 


3. RESULTS 


Of the 44 first-year students, 24 answered the questionnaire in class. We conducted Spearman’s correlation 
analysis to investigate the relationship between SRL, submission time of the one-minute paper and report, 
standard deviation of the one-minute paper’s submission times and the report score. In sections 3.1 and 3.2 
below, we provide the descriptive data, and the results of the correlation analysis are given in section 3.3. 


3.1 Descriptive data and Wilcoxon Signed-Rank Test for SRL 


Table | shows the average, standard deviation, and median results of the Wilcoxon signed-rank test for each 
SRL factor. The score for each factor was calculated from the sum of each item in each factor. Table 2 
displays the descriptive data of the concept map, submission time, and report quality. These results show that 
MSLQ factors except self-efficacy significantly declined between pre- and post-questionnaires. However, 
SDs of internal value, cognitive strategy use, and test anxiety declined very much—that is, factors of MSLQ 
declined overall, but lower-level learner scores improved and individual differences became small. 


3.2 Descriptive data for Learning Behaviors 


Tables 2 and 3 show learning behaviors and learning performance. In BR-Map usage, learners tended to 
create nodes more, though learners used link functions to some extent. With regard to submission of the 
one-minute paper and report, almost all of the learners kept the deadline. Learners tended to submit the 
one-minute paper between noon and evening. The SD of the one-minute paper submission hours indicated 
learning habits. For example, if the SD is 0, it indicates that a learner submits the one-minute paper at the 
same hour every week. In this study, eight out of 24 learners had an SD of submission hour less than 1. 
One-third of the learners had a stable learning habit. Interestingly, in five of the eight learners, the average of 
submission hours for the one-minute paper was less than 10 hours in five lectures. This means that these 
learners submitted their one-minute paper around the end of the day (from 22:00 to 23:59). 
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3.3 Correlation Analysis 


The Wilcoxon signed-ranked test results revealed that awareness of SRL declined significantly overall. 
However, what kinds of learning behaviors and SRL awareness were affected by the use of BR-Map? Is 
awareness of SRL, learning behaviors, and learning performance related to the use of BR-Map? To 
investigate the relationships between psychological perspectives, learning behaviors, learning performance, 
and BR-Map usage, Spearman’s correlation analysis was conducted. The differences between post- and 
pre-rating data for SRL were calculated. Table 4 shows the results. 


Table 1. Average sum scores and Wilcoxon signed-ranked test results for each factor in MSLQ 


Item Average score (SD) Median Z sig 
Pre Post Pre Post 
Self-efficacy (min: 9, max: 31.79 (7.45) 32.29(7.90) 32.00 33.00 -0.20 p = 0.84 
63) 
Internal value (min: 9, 46.71 (11.23) 42.08 (4.93) 49.00 42.00 2.82 p = 0.00 
max: 63) 
Cognitive strategy use 62.50 (12.51) 57.50 (5.99) 61.50 58.00 2.50 p=0.01 
(min: 13, max 91) 
Self-regulation (min: 9, 37.08 (3.45) 33.13 (4.01) 37.50 34.00 3.09 p = 0.00 
max: 63) 
Test anxiety (min: 4, max 16.29 (5.55) 14.00 (1.69) 16.00 14.00 1.83 p = 0.06 
20) 
Table 2. Average sum score of nodes and links in BR-Map 
Item Average score (SD) Median 
Node 8.67 (6.79) 9.00 
Link 3.88 (3.85) 2.50 
Table 3. Average, SD, and median of learning behaviors and learning performance 
Item Average SD Median 
Submission time for one-minute paper in the deadline (Min 4.92 0.28 5.00 
0 — Max: 5) 
Delayed submission time for one-minute paper 0.16 0.38 0.00 
Sum of submission time (hour) for one-minute paper 26.38 17.51 24.00 
Submission time (hour) of S.D. for one-minute paper 2.70 1.89 2.88 
Submission time (hour) for report 20.08 31.18 11.50 
Report score (Min 0 — Max 40) 32.71 4.97 33.00 


Table 4. Spearman’s correlation analysis results between MSLQ, node and link in BR-Map, learning behaviors, and 
learning performance 


Node Link One- One-minute §One-minute S.D of Report Report 
minute paper paper One- submission score 
paper delayed submission minute hours 
submission submission hours paper 

submission 

hours 
SE 0.20 0.11 -0.04 0.06 0.26 -0.01 0.24 0.01 
IV 0.30 0.49* 0.18 -0.07 0.22 0.31 -0.10 -0.04 
CS 0.34 0.44* 0.18 -0.07 0.26 0.367 0.03 0.04 
SR 0.43* 0.47* 0.11 -0.19 0.24 0.12 -0.13 0.11 
TA 0.09 0.10 0.18 -0.11 0.00 0.06 -0.05 0.02 
Node - 0.76*** = 0.42* -0.54** 0.53** 0.19 0.42* 0.22 
Link 0.76*** — - 0.36+ -0.42* 0.41* 0.15 0.15 0.15 
Report 0.22 0.15 0.40+ -0.45* -0.20 -0.10 0.08 - 


ee: D < 0.001, **: p < 0.01, *: p < 0.05, f: p< 0.1 
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The results indicated that self-regulation and the number of links were correlated with the number of 
nodes, and internal value, cognitive strategies use, and self-regulation were correlated with the number of 
links. With regard to the relationships between learning behaviors, learning performance, and BR-Map use, 
the number of nodes was correlated with one-minute paper submissions, one-minute paper submission hours, 
and report submission hours positively, and with one-minute paper delayed submissions negatively. The 
number of links on BR-Map was correlated with one-minute paper submission and one-minute paper 
submission hours positively, and with one-minute paper delayed submissions negatively. However, the usage 
of BR-Map did not have any significant direct relationships with the report score. 


4. CONCLUSION AND FUTURE RESEARCH 


This study aimed to develop and conduct a formative evaluation of BR-Map from the perspective of 
self-regulated learning. It is hypothesized that BR-Map supports SRL skills—in particular, cognitive learning 
strategies use—directly, and it usage seems to be affected by the learning habit of reading learning materials 
regularly. The results of this study support the hypothesis to some extent. The usage of BR-Map was 
significantly correlated with the awareness of self-regulation and the use of cognitive learning strategies in 
MSLQ, and the submission times and hours of the one-minute paper. However, the usage of BR-Map was not 
significantly correlated with learning performance directly. According to many previous studies, supporting 
SRL leads to learning performance (e.g., Wolters et al, 2003; Yamada et al, 2016). There are two possible 
reasons; one is that learners could not effectively use the concept map on BR-Map for report writing. 
BR-Map allows learners to understand learning contents in a cross-class manner, but it did not focus on the 
report theme that the learners wrote. Second, BR-Map seems to enhance the understanding of learning 
materials, but it did not help learners in developing their ideas on the report theme. The class required 
learners to select a report theme and write abstracts of the theme and their idea. When learners did not 
include their idea in their report, it could lead to lower scores. 

As future research, there are four points that need to be taken up. First, to improve BR-Map functions. 
Several learners asked to modify BR-Map, for example, adding memos on BR-Map and displaying the 
thumbnails of learning materials (slides) on the left pane. These functions seem to improve usability and 
affect awareness of learning objectives. Second, analysis of the relationships with ebook viewer logs—such 
as page flipping, highlighting, and memos—is required because these behaviors were considered to have 
direct effects on comprehension and self-regulated learning (Yamada, et al, 2017), as mentioned in section 1. 
Third, to analyze learning behaviors and learning performance with more data in order to investigate the 
effects of BR-Map. And finally, to develop a dashboard to collect and visualize the learning process using 
BR-Map, which would be essential for enhancing the effects of learning analytics on learning support. 
BR-Map is a cognitive medium that connects input and consideration. Learning logs stored on BR-Map seem 
to be useful to understand the learner’s learning process. A dashboard to collect and visualize learning logs 
on BR-Map can be effective in understanding the status of the learner’s learning process, which promotes 
effective learning support. 
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